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DETAILED ACTION 

Continued Examination Under 37 CFR 1.114 
A request for continued examination under 37 CFR 1.114, including the fee set 
forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this 
application is eligible for continued examination under 37 CFR 1.114, and the fee set 
forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action 
has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 
03/15/2006 has been entered. 

Response to Amendment 

The amendment filed 03/15/2006 has been entered and made of record. Claims 
1, 3-8, and 10-14 are pending. 

Response to Arguments 

Hanna et al. (5,566,251) estimates the pose of the detected logo pattern "A" by 
using the orientation, scale and perspective parameters of each of the sequence of 
successive 2D video image frames of the background scene with respect to the image 
of one or more reference objects in that video image frame [col. 4, lines 40-50]. 
Additionally, the system/method of Hanna performs the function of geometrically 
transforming the orientation and scale of logo pattern "B" of foreground scene to match 
the orientation and scale of the estimated pose of logo pattern "A" [col. 4, line 64 - col. 5 
line 12]. Therefore Hanna teaches processing the user information as dependent on 
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the orientation, scale and perspective (said size alteration of the image information). 
Furthermore, Applicant fails to communicate how Hanna in view of Silverbrook fails to 
teach the limitations of Applicant's invention. Therefore, from the response above, 
Hanna in view of Silverbrook further teaches the newly amended limitation of claim 8, as 
well as from the office action below, the limitations of claims 1 , 3-8, and 10-14. 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill In the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the Invention was made. 

1. Claims 1, 3-8, and 10-14 are rejected under 35 U.S.C. 103(a) as being 

unpatentable over Hanna et al. (US Patent No. 5,566,251) in view of Silverbrook (US 

Patent No. 5,479,205). 

Hanna et al. teaches the limitation of claims 1, 3-8, and 10-14 with the exception of 

disclosing a triggering unit and a display screen. However, Silverbrook discloses a 

video camera and recorder for combining a real-time image with a secondary image. 

In regards to claims 1, 7, 8 and 14 - 

Referring to Fig. 2, camera 200A records in real time a sequence of successive 2- 
dimensional video image frames of physical three-dimensional objects, such as 
billboard (202), situated in background scene (204A) (claims 1, 8: a camera configured 
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to acquire image information of a section of an environment) [col. 4, lines 18-22]. 
Billboard (202) is assumed immovable, but camera (200A) is moveable both to the left 
and right and toward and away from the objects as indicated by arrowed lines (206A). 
This will result in the orientation and scale of each of the sequence of video images of 
billboard (202) in background scene (204A) changing in correspondence with the 
movement of camera (200) from one position to another, as indicated by arrow lines 
(208A) (claims 1, 8: af least one of a zoom device configured to altera size of the 
section in accordance with a zoom factor and a device configured for three-dimensional 
orientation of the a camera in accordance with a space vector) [col. 4, lines 24-33]. 
Means 21 OA performs the functions of (1) detecting logo pattern "A" and (2) estimating 
the pose of detected logo pattern "A" (i.e., estimating the orientation, scale and 
perspective parameters of logo pattern "A" in each of the sequence of successive 2D 
video image frames of background scene (204A) with respect to the image of one or 
more reference objects, such as billboard (202) itself) in that video image frame (claims 
1,8: wherein the computer unit is configured to compute position coordinates of the 
image information based on at least one of space coordinates of the camera, the zoom 
factor, and the space vector, computer positions of representations of the image 
information on a display area of a display device] claims 7, 14: processing image 
information) [col. 4, lines 43-49]. Additionally, means 214, responsive to a "pose 
information" (i.e., orientation, scale, perspective distortion, etc. parameters of detected 
logo patter "A") input (215) from means 21 OA and on-time selection via input (217) of 
manually-selected size, shape, orientation, etc. parameters of logo pattern "B" itself 
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performs the function of geometrically transfonning the orientation and scale of logo "B" 
of foreground scene (204B) to match the orientation and scale of the estimated pose of 
logo pattern "A" in the current image frame (as indicated by (216)) (claims 1, 8: wherein 
the computer unit is configured to assign the user information to the position 
coordinates] claim 8: wherein at least one of the display area and the processing of the 
user information is dependent upon the size alteration; claims 7, 14: processing use 
information) [col. 4, lines 64-68; col. 5, lines 1-6]. The operation of video switch (212) 
merges the video output of camera (200) with the geometrically-transformed logo 
pattern "8" to result in output scene (218), wherein logo pattern "A" is replaced by 
inserted logo pattern "B" (claims 1 , 8: an image processing unit for processing the 
image infonvation and the user information so as to reproduce the image information 
and the user information with the display device and so as to insert the user information 
in a location on the display area at the positions of the representations of the image 
information having the position coordinates to which the respective user information is 
assigned] claims 7, 14: reproducing the image information and the user information with 
the display device, inserting the user information in the location on the display area in 
accordance with an imaging procedure) [col. 5, lines 6-9]. 

Silverbrook discloses a camcorder system. Referring to Fig. 1 , the video 
camcorder system's circuitry (40) includes an alpha-numeric liquid crystal display (LCD) 
(61) and various miscellaneous controls (62) such as an auto zoom controller (66), and 
a manual zoom control (70) as seen in Fig. 2 (claims 1, 8: comprises a triggering unit 
configured to trigger at least one of the camera, the zoom device, and the device for 3D 
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orientation of the camera is accordance with at least one of the zoom factor and the 
space vector) [col. 2, lines 23-30]. The circuitry (40) of the system receives user input 
via a keyboard (17) that includes an alphanumeric keypad (33) a touch panel (31), and 
animation control keys (38) as shown in Fig. 2. The keyboard (17) connects to a 
controller (16) that interprets commands and provides control commands onto a bus 
(25) that interconnects other components within the system (40) [col. 2, lines 43-45]. 

Therefore, it would have been obvious to one of ordinary skill in the art at the time 
the invention was made to substitute the camera of Hanna et al. with the video 
camcorder system of Silverbrook where the output from the CCD of Silverbrook would 
be connected to means 21 OA of Hanna et al. so that the orientation and scaling of the 
camera within the combined invention is able to be user controlled. 

In regards to claims 3 and 10 - 

The rationale of claims 1 and 8 respectively, are incorporated herein. Referring to 
Fig. 2 of Hanna et al., camera (200A) is moveable both to the left and right and toward 
and away from the objects as indicated by arrowed lines (206A). This will result in the 
orientation and scale of each of the sequence of video images of billboard (202) in 
background scene (204A) changing in correspondence with the movement of camera 
(200) from one position to another, as indicated by arrow lines (208A) [col. 4, lines 24- 
33]. Thus the movement of the camera toward the billboard (202) in the background 
scene (204A) and the scaling correspond to a zoom function. Means 21 OA performs 
the functions of (1) detecting logo pattern "A" and (2) estimating the pose of detected 
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logo pattern "A" (i.e.. estimating the orientation, scale and perspective parameters of 
logo pattern "A" in each of the sequence of successive 2D video image frames of 
background scene (204A) with respect to the image of one or more reference objects, 
such as billboard (202) itself) in that video image frame [col. 4, lines 43-49]. 
Additionally, means 214. responsive to a "pose information" (i.e., orientation, scale, 
perspective distortion, etc. parameters of detected logo patter "A") input (215) from 
means 210A and on-time selection via input (217) of manually-selected size, shape, 
orientation, etc. parameters of logo pattern "B" itself performs the function of 
geometrically transforming the orientation and scale of logo "B" of foreground scene 
(204B) to match the orientation and scale of the estimated pose of logo pattern "A" in 
the current image frame (as indicated by (216)) (claims 3, 10: configured to select and 
insert tlie user information as a function ofttie zoom factor) [col. 4, lines 64-68; col. 5, 
lines 1-6]. 

In regards to claim 4 and 11 - 

The rationale of claims 1 and 8 respectively, are incorporated herein. Referring to 
Fig. 2 of Hanna et al., the operation of video switch (212) mergers the video output of 
camera (200) with the geometrically transformed logo pattern "B" (i.e. user information 
as claimed) to result in output scene (218), wherein logo pattern "A" is replaced by 
inserted logo pattern "B". Thus, the pose of logo pattern "B" in output scene (218) 
changes in correspondence with the movement of camera (200) from one position to 
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another, as indicated by arrowed lines (220) (claims 4, 11: user information comprises 
at least one of static and dynamic information) [col. 5, lines 5-12]. 

In regards to claims 5 and 12 - 

The rationale of claims 1 and 8 respectively, are incorporated herein. Silverbrook 
discloses a camcorder system. Referring to Fig. 1 . the video camcorder system's 
circuitry (40) includes an alpha-numeric liquid crystal display (LCD) (61) (claims 5, 12: 
the camera comprises a video camera and ttie display device comprises a display 
screen) and various miscellaneous controls (62) such as an auto zoom controller (66), 
and a manual zoom control (70) as seen in Fig. 2 [col. 2, lines 23-30]. 

In regards to claims 6 and 13 - 

The rationale of claims 1 and 8 respectively, are incorporated herein. Silverbrook 
discloses a camcorder system. Referring to Fig. 1 , the video camcorder system's 
circuitry (40) includes an alpha-numeric liquid crystal display (LCD) (61) and various 
miscellaneous controls (62) such as an auto zoom controller (66), and a manual zoom 
control (70) as seen in Fig. 2 {said triggering unit) [col. 2, lines 23-30]. The circuitry 
(40) of the system receives user input via a keyboard (17) that includes an 
alphanumeric keypad (33) a touch panel (31). and animation control keys (38) as shown 
in Fig. 2. The keyboard (17) connects to a controller (16) that interprets commands and 
provides control commands onto a bus (25) that interconnects other components within 
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the system (40) (claims 6, 13: the triggering unit comprises an operator interface) [col. 
2, lines 43-45]. 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Michelle K. Lay whose telephone number is (571) 272- 
7661 . The examiner can normally be reached on Monday through Thursday from 
7:30am to 5:00pm. The examiner can also be reached on alternate Fridays. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor. Kee M. Tung, can be reached at (571) 272-7794. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 
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